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DETAILED ACTION 

Continued Examination under 37 CFR 1.1 14 

1. A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 
CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for 
continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1 .17(e) has been timely 
paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. 
Applicant's submission filed on March 10, 2010 has been entered. 

Response to Amendment 

2. Applicant's amendment filed on March 10, 2010 has been considered and entered in full. 

3. Applicant's amendment to the claims have been considered and entered in full. Based on 
the applicant's amendment to the claims, claim rejections under 35 USC 112 have been withdrawn. 

4. Applicant's arguments with respect to the claims have been considered but are not 
persuasive. 

Response to Arguments 

5. Applicant's arguments regarding the prior art under Hatfield and Newman on pages 10-12 of 
the amendment filed on March 10, 2010 have been fully considered but are not persuasive. 
Applicant argues in substance that: 

"Hatfield discloses a user interface that utilizes both eye gaze data and voice recognition 
data. This allows a single user to interact with a computer system. Newman is cited to teach a system 
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that can be responsive to multiple users. However, Hatfield is used to teach cursor control based on 
user eye movement. A cursor cannot be controlled by multiple user's eye movements because each 
user would move their eyes differendy rendering the cursor useless. Newman is similarly directed to 
response to eye input from a single user. Specifically, Newman is directed to a wearable computer. 
See Figure 1. As with Hatfield, input from multiple users would render the device of Newman 
useless. The Office Action states that the references show input from one or more users. However, 
there is no citation to that teaching on either reference. Thus, Applicants submit that both 
references teach away from use of eye tracking data from multiple users as explicitly recited in the 
claims. Therefore, no combination of references can teach or suggest or suggest the invention as 
recited in the claims". 

Examiner respectfully disagrees. Examiner provided a note for the applicant which said 
"Examiner has cited particular columns and line numbers in the references as applied to the 
claims above for the convenience of the applicant. Although the specified citations are 
representative of the teachings for the art and are applied to the specific limitations within the 
individual claim, other passages and figures may be applied as well. It is respectfully requested 
from the applicant in preparing responses, to fully consider the references entirely as potential 
teachings all or part of the claimed invention, as well as the context of the passage as taught by 
the prior art or disclosed by the examiner" and applicant has completely ignored the note in 
preparing the response. Examiner has clearly cited the subject matter from Newman on page 7 of 
the office action mailed on 11/10/2009 which discloses the teachings of use of a host computer 
by one or more users. Examiner here again cites the following from Newman: 
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Newman discloses a handsfree system in which the user may operate the computing 
apparatus to display information in a hands-free manner utilizing eye tracking with or without 
audio commands (col. 2, lines 61-64). Newman further discloses "the system concept of eye- 
tracking permits the operator to manipulate the computer completely hands-free by tracking the eye 
and interpreting its movements as "mouse" commands to the application. Generally, the eye 
tracking system is head-mounted on the user but may be used in any suitable manner" (col. 3, lines 
40-45). Newman further discloses "a voice command can be given to the present computer which is 
confirmed or enhanced by the present eye-tracking system and vice- versa" (col. 3, lines 42-45). 

Newman further discloses "The remote computer can be in contact with several users 
concurrently each having activation means, headset with display means, and 
communication means to interact with the host computer. Therefore, in this second mode, 
it is possible, for example, to permit several users with headsets to function off one portable 
mobile host computer carried by one of several users either working together or at remote 
locations" (col. 6, lines 16-24). Clearly, each user has its own display and activation means, which 
let them operate individually and simultaneously. Newman further discloses "the system can work 
in two modes: In the first mode, a user having a headset with activation means, display screen and 
communication means is linked with a host computer either worn on the user's person contained 
within the display means or remote from the person. The user would command the host computer 
to display a particular procedure for repairing a piece of equipment. The host computer would then 
search for an transmit the procedure back to the user for display on the user's headset. In the 
second mode, a host computer could also be responsive to several users having headsets 
that may be working as a team to repair the piece of equipment. The users could all work 
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from the same procedure or call for a different procedure to be displayed as an additional 
window in the display (col. 7, lines 1-15). 

Further adding, on analysis of applicant's arguments it seems applicant is arguing 
simultaneous use of the system by multiple users, which the claims does not even claim. However, 
the subject matter of simultaneous use of the system by multiple users has been taught by Newman. 
Therefore, Office still maintains the rejections on the respective claims. 

Claim Rejections - 35 USC § 101 

6. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any 
new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of 
this title. 

The USPTO "Interim Guidelines for Examination of Patent Applications for Patent Subject 
Matter Eligibility" (Official Gazette notice of 22 November 2005 
<http://www.uspto.gov/web/offices/ com/ sol/ og/ 2005/week47/og200547.htm>), Annex IV, 
reads as follows: 

In contrast, a claimed computer-readable medium encoded with a computer program is a 
computer element which defines structural and functional interrelationships between the 
computer program and the rest of the computer which permit the computer program's 
functionality to be realized, and is thus statutory. See Lowry, 32 F.3d at 1583-84, 32 USPQ2d at 
1035. 
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Claims that recite nothing but the physical characteristics of a form of energy, such as a 
frequency, voltage, or the strength of a magnetic field, define energy or magnetism, per se, and 
as such are nonstatutory natural phenomena. O'Reilly, 56 U.S. (15 How.) at 112-14. Moreover, it 
does not appear that a claim reciting a signal encoded with functional descriptive material falls 
within any of the categories of patentable subject matter set forth in Sec. 101. 

... a signal does not fall within one of the four statutory classes of Sec. 101. 

. . . signal claims are ineligible for patent protection because they do not fall within any of the 

four statutory classes of Sec. 101. 
The USPTO "Interim Guidelines for Examination of Patent Applications for Patent Subject 
Matter Eligibility" (Official Gazette notice of 23 February 2010 
<http://www.uspto.gov/web/ offices/ com/ sol/ og/ 2010/week08/TOC.htm#ref20>), reads as 
follows: 

"The USPTO recognizes that applicants may have claims directed to computer readable 
media that cover signals per se, which the USPTO must reject under 35 U.S.C. § 101 as covering 
both non-statutory subject matter and statutory subject matter. In an effort to assist the patent 
community in overcoming a rejection or potential rejection under 35 U.S.C. § 101 in this situation, 
the USPTO suggests the following approach. A claim drawn to such a computer readable medium 
that covers both transitory and non-transitory embodiments may be amended to narrow the claim to 
cover only statutory embodiments to avoid a rejection under 35 U.S.C. § 101 by adding the 
limitation "non-transitory" to the claim. Cf. Animals - Patentability, 1077 Off. Gaz. Pat. Office 24 
(April 21, 1987) (suggesting that applicants add the limitation "non-human" to a claim covering a 
multi-cellular organism to avoid a rejection under 35 U.S.C. § 101). Such an amendment would 
typically not raise the issue of new matter, even when the specification is silent because the broadest 
reasonable interpretation relies on the ordinary and customary meaning that includes signals per se. 
The limited situations in which such an amendment could raise issues of new matter occur, for 
example, when the specification does not support a non-transitory embodiment because a signal per 
se is the only viable embodiment such that the amended claim is impermissibly broadened beyond 
the supporting disclosure. See, e.g., Gentry Gallery, Inc. v. Berkline Corp., 134 F.3d 1473 (Fed. Cir. 
1998)." 



7. 



Claims 9-16 and 20-22 are rejected under 35 U.S.C. 101 because: 
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Claims 9-16 and 20-22 are drawn towards a computer-readable medium having a computer 

program or instructions embodied thereon. Such computer executable programs are functional 

descriptive material that can constitute statutory products when properly claimed in combination 

with a computer readable storage device that allows the functionality of the program to be realized. 

However, while the claim defines the program in combination with a "computer usable medium," 

the claim language is not limited to the use of statutory storage media in combination with the 

program. Specifically, the disclosure defines the computer accessible/ readable medium as including 

read only memory (ROM); random access memory (RAM); magnetic disk storage media; optical 

storage media; flash memory devices; electrical, optical, acoustical or other form of propagated 

signals (e.g., carrier waves, infrared signals, digital signals) (paragraph 0020 on page 9 of the 

specification). While the read only memory (ROM); random access memory (RAM); magnetic disk 

storage media; optical storage media; flash memory devices (statutory media) are physical storage 

devices, but the carrier wave/infrared signal/digital signal is a signal which is intangible energy, and 

therefore is not considered to be a statutory product. While the claim does read on at least some 

statutory embodiments (such as the floppy or hard or magnetic or optically readable disks), it also 

encompasses the combination of the program with the carrier wave, which is not a statutory 

product. Since the claimed invention encompasses non-statutory embodiments, the claim is directed 

towards non-statutory subject matter. 

Claims 9-16 and 20-22 as a whole define a carrier wave, and "[a] transitory, propagating signal 

is not a "process, machine, manufacture, or composition of matter." Those four categories 

define the explicit scope and reach of scope and reach of subject matter patentable under 35 
U.S.C § 101; thus, such a signal cannot be patentable subject matter." (In re PetmsA.CM. Nutijen; 
fed Cir, 2006-1 371, 9 1 20 1 2007). 

In an effort to assist the patent community in overcoming a rejection or potential rejection 
under 35 U.S.C. § 101 in this situation, the USPTO suggests the following approach. A claim drawn 
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to such a computer readable medium that covers both transitory and non-transitory embodiments 
may be amended to narrow the claim to cover only statutory embodiments to avoid a rejection 
under 35 U.S.C. § 101 by adding the limitation "non-transitory" to the claim. ). Such an amendment 
would typically not raise the issue of new matter, even when the specification is silent because the 
broadest reasonable interpretation relies on the ordinary and customary meaning that includes 
signals per se. The limited situations in which such an amendment could raise issues of new matter 
occur, for example, when the specification does not support a non-transitory embodiment because a 
signal per se is the only viable embodiment such that the amended claim is impermissibly broadened 
beyond die supporting disclosure. See, e.g., Gentry Gallery, Inc. v. Berkline Corp., 134 F.3d 1473 
(Fed. Cir. 1998)." 



Claim Rejections - 35 USC § 103 

8. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 
rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identical!) disclosed or described as set forth in section 
102 of this title, if the differences between ihe subject matter sought to be patented and the prior art are such that the 
subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary 
skill in the art to which said subject matter pertains. Patcntabiliu shall not be negatived by the manner in which the 

9. Claims 1-2, 4-5, 7-10, 12-13 and 15-22 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Hatfield et al., November 30, 1997, "An Interface integrating eye gaze and voice 
recognition for hands-free computer access" (pages 1-7), further in of view of Newman et al., U.S. 



Patent No. 5,844,824. 
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Hatfield discloses an EVA (Eye/Voice Aware) interface which allows combined eye-tracking 
and voice input to allow hands-free interaction with a standard window-based computer application, 
where voice input being the external context data. 

Hatfield discloses "In EVA, we implemented a mode of operation where the user can slave the 
screen cursor to his/her eye. Wherever the user looks on the display while operating in this mode, 

the cursor will be displayed. . .The user fixates a Windows button The system will detect rapid 

eye movement, e.g., saccades and adjust very quickly to gross changes in POG "(Hatfield, topic 4 - 
Initial EVA Implementation, pages 4 and 5). Clearly from Hatfield's disclosure, it is clear that the 
cursor is a slave of the eye, and the cursor motion is defined by the eye movement (or eye tracking 
data) and it being apparent that in order to interpret the eye motion (or eye tracking data), an eye 
interpretation engine (or system or component) is inherendy required which would send the 
translated data to the system to synchronize the cursor with the eye motion (or eye behavior), 
without which EVA system won't be able to synchronize the eye with cursor. The interpretation 
component is a must part of a system which uses eye motion or eye behavior to control a system. 
Thus, EVA system, receiving from an eye interpretation engine, at least an interpretation of 
eye tracking data corresponding to at least one user. 

Hatfield further discloses receiving external context data corresponding the at least one 
user's interaction with an application (Topic 4 - Initial EVA Implementation- pages 4 and 5 - 
external context data being voice commands such as "forward tab", "click", "page up" and "page 
down" — With IE users can navigate the various hyperlinks in a web page by "tabbing between 
them" with the command "forward tab". They can also fixate a particular hyperlink and utter 
"click", page 4 -last few lines). Now, as in Microsoft Windows, the operations of the computing 
device running the application are controlled by the use of cursor and the buttons, and controlling 
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the system is basically controlling the operational state of the processor of the computing device. 
Hatfield discloses that voice input is used as command entry for controlling the operational states of 
the processor of the computer running the application. For example, voice commands such as 
"forward tab, "page up" and "page down" would control the operational state of browsing of 
internet explorer in the device (pages 4, last few lines through page 5), which conforms to the 
limitation "wherein the external context data is related to an operational state associated with a 
computing device running the application". 

Hatfield further discloses "based, at least in part, on the interpretation of the eye tracking 
data and the external context data, dynamically modifying a characteristic of a graphical user 
interface of the application" (Topic 4 - Initial EVA Implementation - page 4. last few lines through 
page 5 - in internet explorer, fixate a particular hyperlink and utter "click" which has the effect of 
clicking with a mouse on the hyperlink, thus hyperlink being the characteristic of the application 
(internet explorer) and dynamically acting on the characteristic being clicking on the hyperlink to 
invoke the internet explorer to display the contents of the link on the browser and after clicking, it 
opens a new link in the browser which being dynamically modifying a characteristic of a graphical 
user interface of the application). 

As discussed before, Hatfield discloses an EVA (Eve/Voice Aware) interface which allows 
combined eye-tracking and voice input to allow hands-free interaction with a standard window- 
based computer application, where voice input being the external context data. Hatfield provides an 
example of an application in which the combination of eyetracking and voice commands is used by a 
single user of the computer but does not expressly teach the usage of the combination of 
eyetracking and voice by a plurality of users. 
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However, Newman discloses a hands free system in which the user may operate the 
computing apparatus to display information in a hands-free manner utilizing eye tracking with or 
without audio commands (col. 2, lines 61-64). Newman further discloses "the system concept of 
eye-tracking permits the operator to manipulate the computer completely hands-free by tracking the 
eye and interpreting its movements as "mouse" commands to the application. Generally, the eye 
tracking system is head-mounted on the user but may be used in any suitable manner" (col. 3, lines 
40-45). Newman further discloses "a voice command can be given to the present computer which is 
confirmed or enhanced by the present eye-tracking system and vice-versa" (col. 3, lines 42-45). 
Newman further discloses "The remote computer can be in contact with several users 
concurrently each having activation means, headset with display means, and 
communication means to interact with the host computer. Therefore, in this second mode, 
it is possible, for example, to permit several users with headsets to function off one portable 
mobile host computer carried by one of several users either working together or at remote 
locations" (col. 6, lines 16-24). Clearly, each user has its own display and activation means. 
Newman further discloses "the system can work in two modes: In the first mode, a user having 
a headset with activation means, display screen and communication means is linked with a host 
computer either worn on the user's person contained within the display means or remote from the 
person. The user would command the host computer to display a particular procedure for repairing 
a piece of equipment. The host computer would then search for an transmit the procedure back to 
the user for display on the user's headset. In the second mode, a host computer could also be 
responsive to several users having headsets that may be working as a team to repair the 
piece of equipment. The users could all work from the same procedure or call for a different 
procedure to be displayed as an additional window in the display" (col. 7, lines 1-15). 
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Therefore, it would have been obvious to one of ordinary skill in the art at the time invention was 
made to combine the teachings of Newman of extending the combination of eye tracking and voice 
data to multiple-users to control/ operate the computer with that of Hatfield. One of ordinary skill in 
the art would have been motivated to combine the teachings of Newman with that of Hatfield 
because both references are directed to the same field of endeavor and Newman's teachings when 
combined with the teachings of Hatfield would enhance or provide the Hatfield's invention with 
additional capability or versatility of handling applications where multiple users concurrently use the 
computer to share the same application when working on the same task (especially when different 
users are accessing a host computer from their individual activation and display means), and would 
further increase the efficiency of the group or users working on the task. 

Regarding claim 2, claim 2 recites wherein acting on comprises one of: modifying, creating, 
destroying, removing, invoking and configuring. Hatfield discloses "in internet explorer, fixate a 
particular hyperlink and utter "click" which has the effect of clicking with a mouse on the 
hyperlink", (Topic 4 - Initial EVA Implementation - page 4. last few lines through page 5), thus 
hyperlink being the characteristic of the application (internet explorer) and dynamically acting on the 
characteristic being clicking on the hyperlink to invoke the internet explorer to display the contents 
of the Hnk on the browser. 

Regarding claim 4, all the limitations in the claim 4 are similar to claim 1 except that claim 4 
recites "dynamically acting on a graphical output of the application displayed on an output device 
based, at least in part, on the interpretation of the eye tracking data and the external context data". 
As discussed before, Hatfield discloses (Topic 4 - Initial EVA Implementation - page 4. last few 
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lines through page 5 — in internet explorer, fixate a particular hyperlink and utter "click" which has 
the effect of clicking with a mouse on the hyperlink) thus hyperlink being the characteristic of the 
application (internet explorer) and dynamically acting on the characteristic being clicking on the 
hyperlink to invoke the internet explorer to display the contents of the link on the browser. Thus 
clicking on the hyperlink would invoke the internet explorer to display the contents of the link as 
output on the browser, where the output again being a page in the internet explorer with hyperlink. 
Hatfield further discloses acting on the output (page 5, topic 4, 1 st few lines - Users can also scroll 
the browser by issuing "page up" and "page down" commands, open list boxes and select various 
items). 

Claim 5 has been similarly analyzed and rejected as per claim 2. 

Claims 7-10, 12-13 and 15-16 have been similarly analyzed and rejected as per claims 1-2 and 

4-5. 

Regarding claims 17-22, Hatfield discloses the external sources (external context data) 
comprise one or more of: system information, location and/or z-order of windows and/or objects, 
document Object Model (DOM) of a Web page or application being viewed, current application 
process state and/or visual state, task models, cognitive models describing the mental or physical 
steps or states required (page 4 - last few lines - current application process state and/or visual 
state - tabbing between the hyperlinks which being the process/ visual state of browsing, the tasks 
model including tabbing and scrolling, fixating and clicking a windows button). 
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10. Claims 3, 6, 11 and 14 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Hatfield et al., November 30, 1997, "An Interface integrating eye gaze and voice recognition for 
hands-free computer access" (pages 1-7), further in of view of Newman et al., U.S. Patent No. 
5,844,824, and further in view of Martins, U.S. Patent No. 6,608,615 Bl. 

Regarding claims 3, 6, 11 and 14, as discussed in the rejection of claims 1 and 4, the 
combined invention of Hatfield and Newman discloses that eye tracking data can be used to control 
various aspects of the interface for example controlling the browsing in internet explorer (IE) and 
selecting the various items in IE but combined invention of Hatfield and Newman does not 
explicitly disclose the extended applications where the eye tracking data would be used such as 
"determining from the interpretation of the eye tracking data at least a portion of an interface that 
has not been viewed by the user and modifying a format of the portion of the interface that has not 
been viewed by the user". However, Martins which belong to the same field of endeavor describes 
the extended applications where the eye tracking data can be used. Martins discloses "if the banner 
add is not viewed at all, the intervention would cause the banner to dynamically appear to jump 
closer to the user's gaze" (Martins, col. 6, lines 19-21). Therefore, it would have been obvious for 
one of ordinary skill in the art at the time of invention was made to combine the teachings of 
Martins with that of combined invention of Hatfield and Newman. One of ordinary skill in the art 
would have been motivated to combine the teachings of Martins with that of combined invention of 
Hatfield and Newman as it would enhance the capabilities of combined invention of Hatfield and 
Newman to be used in extended applications, thus making Hatfield's system more versatile in the 
field of controlling the interface application using eye tracking data. 
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Examiner note: Examiner has cited particular columns and line numbers in the references as 
applied to the claims above for the convenience of the applicant. Although the specified citations are 
representative of the teachings for the art and are applied to the specific limitations within the 
individual claim, other passages and figures may be applied as well. It is respectfully requested from 
the applicant in preparing responses, to fully consider the references entirely as potential teachings 
all or part of the claimed invention, as well as the context of the passage as taught by the prior art or 
disclosed by the examiner. 



Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Manav Seth whose telephone number is (571) 272-7456. The examiner can 
normally be reached on Monday to Friday from 8:30 am to 5:00 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 
Vikkram Bali, can be reached on (571) 272-7415. The fax phone number for the organization where 
this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR system, 
see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, 
contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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/Manav Seth/ 

Primary Examiner, Art Unit 2624 
March 25, 2010 



